Repository navigation
fix(cache): retain execution backing and bound external cache transfers - #3
Merged
Merged
Conversation
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
feichai0017
force-pushed
the
feat/cache-identity-budgets
branch
from
November 15, 2026 11:19
43afdd2 to
9655934
Compare
feichai0017
force-pushed
the
feat/cache-identity-budgets
branch
from
November 15, 2026 11:43
9d482fe to
7fe76c8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
External prefix keys previously omitted weights, layout and shard authority; queued transfers also had no borrowed-byte ceiling. More seriously, a poisoned Engine retained logical pages but dropped the actual runner and completion buffers at final-reference destruction. The deterministic controls fail on the old implementation at the backing weak-reference assertion.
This binds explicit
CacheIdentityplus ordered cache geometry, validates all exports before submission, and retains byte charges through retirement. The DRAM reference provider now bounds queued jobs and bytes. Native EngineCore requires an actual backing owner; Python supplies its runner and retains uncertain completions and temporary failure frames. Successful close still releases normally. No second scheduler or page allocator is added.Breaking development APIs: native
EngineCore::newtakes the backing Arc; external Python storage requiresprefix_cache_identityandprefix_cache_max_bytes. Old external keys intentionally miss. These are recorded in releases.md.Validation:
uv run --no-sync python scripts/check.pypassed on the combined #3/#4 tree (311 Rust controls; 2,461 Python passed, 551 explicit device/platform skips; packaged CPU example and all three model packages). Ruff, ty and pinned Clippy pass. Receipt/identity/owner controls passed 50 repetitions; all seven execution-drop controls are in the existing 50x CI step. New SM100 gate is written, not run. GPU/model correctness remains unqualified.The first Linux run exposed an additional CPU-fixture shutdown gap (
fatal-2, SIGABRT). A deterministic control proves thatdestroy_process_groupleaves Gloo alive when the intentionally quarantined runner retains its handles. In pinned Torch 2.13.0, the Gloo shutdown hook is inherited and thread joins occur in the destructor. CPU-only fixtures now detach retired communication handles after synchronous work and serving/watch threads finish, assert destruction, and still retain the poisoned model backing. This also covers the multiphase distributed fault fixture. Removing the detach makes the new control fail at the live-native-owner assertion. Both the forced control and real two-rank fatal HTTP case passed 50 local repetitions and run 50x in CI. The standalone CI repetition step also uses the same explicit loopback interface as scripts/check.py (lo0 on macOS, lo on Linux); it must not depend on runner hostname routing. Actionlint passes. The complete 59-control combined tree passed 50 local repetitions (2,950 executions). This is consistent with the upstream Gloo shutdown race; no CUDA teardown guarantee is inferred from it.Handoff:
uv run --no-sync python tests/hardware/cache_storage_cuda.py --device cuda:0(SM100, unrun).